Fixing Fragment Shader Silent Optimization Issue (Issue #284)

This PR addresses Issue #284, where the fragment shader successfully compiles but is silently optimized to have no output operations, leading to silent rendering without errors or warnings, making debugging very difficult.

✅ Use --allow-fragment-no-output to optionally skip ERROR

After adding this feature, many of the previous test cases failed in Github CI because they were completely optimized out by DCE (Dead Code Elimination). After analysis, marking fragment shaders with no output as errors indeed leads to unnecessary test failures. Based on the maintainer's suggestion, a compiler flag was added to control this check. You can optionally skip the error by using --allow-fragment-no-output.

Root Cause Analysis

After a thorough analysis of DCE, we found that Issue #284 is actually a "false scenario," based on the following two core facts:

1. Entry-point functions are never deleted: In the current DCE implementation, all functions listed in the module's OpEntryPoint are unconditionally marked as "root," making them reachable rooted objects. Even if all instructions in the function body are identified as pure calculations and removed, the function definition and OpEntryPoint declaration are retained. DCE does not remove the entire entry-point function.

2. The "silently compiling out" phenomenon is actually the simplification of the function body: When no "obvious output" is written, DCE removes internal computations (such as OpCompositeConstruct, temporary OpLoad, etc.) and leaves only a direct OpReturn. However, it does not remove %main_fs = OpFunction... or OpEntryPoint Fragment %main_fs "main_fs" .... In the SPIR-V module generated by the compiler, the fragment shader still exists—it just no longer writes to the Output variable.

Thus:

• The phenomenon in Issue Fragment shader can be silently compiled out #284, where "the entire fragment shader is silently compiled out," is a misunderstanding of the behavior: "function body is simplified to not write outputs" → "no color is written."

• The initially proposed "silent compile-out" didn’t actually happen: The SPIR-V still contains empty functions and entry-point declarations; it's just that nothing is rendered during the rendering phase.

Solution

Part A — Add Error for Fragment Shaders with "Empty Output"

To fix this, we need to add a check/error for fragment shaders with "empty outputs," so users can immediately know: "Hey, this fragment shader doesn't write anything to the output, it’s definitely a bug," rather than generating an invalid SPIR-V that "renders nothing."

Therefore, I added a validation step after DCE to check for fragment shaders with no output operations:

• validate_fragment_shader_outputs(): The main validation function called after DCE.
• fragment_shader_writes_output(): Recursively checks for output operations.
• has_partial_output_writes(): Detects evidence of partial writes that were optimized away.
• is_output_storage_variable(): Identifies output variables in different scopes.

Part B — Fix CodegenArgs::from_session Comma-Separated Parameter Parsing Issue

During testing, we also discovered that CodegenArgs was incorrectly parsing comma-separated parameters, such as:

// compile-flags: -C llvm-args=--disassemble-entry=main,--allow-fragment-no-output

So, the code at crates/rustc_codegen_spirv/src/codegen_cx/mod.rs:340 was fixed to:

let expanded_args: Vec<String> = sess.opts.cg.llvm_args
    .iter()
    .flat_map(|arg| arg.split(',').map(|s| s.to_string()))
    .collect();

Part C — Add --allow-fragment-no-output to Optionally Skip ERROR

All relevant test cases involving "no output fragment shaders" have now been marked with this flag. Theoretically, they should pass in Github CI. However, there are still two failing tests locally (which appear unrelated to my changes):

• ui/spirv-attr/invalid-target.rs: JSON output too large, causing parsing errors (pre-existing issue).
• ui/spirv-attr/all-builtins.rs: SPIR-V validator strictness issue (environment-related).

Example Error Message

I added a test case that will always fail to verify that silent failures trigger an error message:

• tests/compiletests/ui/dis/issue-284-dead-fragment.rs: A fragment shader with no output (correctly fails and provides a clear error).

Before the fix (silent failure): The shader compiled successfully but rendered nothing.

After the fix (clear error and guidance):

error: fragment shader `main_fs` produces no output
  |
  = help: fragment shaders must write to output parameters to produce visible results
  = note: use complete assignment like `*out_frag_color = vec4(r, g, b, a)` instead of partial component assignments
  = note: partial component assignments may be optimized away if not all components are written
  = note: to disable this validation (e.g. for testing), use `--allow-fragment-no-output`

Since the validation only runs after DCE, does not impact the hot compilation path, and only checks fragment shaders (not vertex/compute shaders), its performance impact is minimal.